feat: extra build artifacts #613

mistydemeo · 2023-11-28T00:09:06Z

This adds support for specifying arbitrary extra artifacts alongside the primary builds. These artifacts are currently expected to be architecture-independent and are uploaded as their own build artifacts rather than bundled into release tarballs. For example, we can use this to replace our current append-release code in to generate and upload the dist-manifest-schema.json.

I realized that most of our generic build code is very well-suited to this, so I split the existing generic builds into half. The code that runs the builds is now shared between this and the generic build path, while another section is separate for implementation detail reasons. There are two main distinctions between these and generic builds:

Generic builds are oriented around binaries, which will eventually be collated into an artifact. Extra artifacts produce artifacts directly, and have no associated binaries.
Generic builds (usually) build native code and have target triples; as a result, we get N copies of the generic build, one per desired triple. Extra artifacts do not have target triples, and we produce a single copy per release.

The syntax looks like this:

[[workspace.metadata.dist.extra-artifacts]]
artifacts = ["dist-manifest-schema.json"]
build = ["cargo", "dist", "manifest-schema", "--output=dist-manifest-schema.json"]

There can be arbitrary numbers of extra artifacts per build, so this is repeatable within a Cargo.toml/dist.toml as many times as needed.

Closes #349.

Gankra

Seems right, just a question of the scope of the config

Gankra · 2023-11-28T19:35:08Z

.github/workflows/release.yml

@@ -59,7 +59,7 @@ jobs:
        with:
          submodules: recursive
      - name: Install cargo-dist
-        run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.5.0/cargo-dist-installer.sh | sh"
+        run: "cargo install --git https://github.com/axodotdev/cargo-dist/ --branch=extra-artifacts cargo-dist"


oroboros....

Haha, yeah, just for testing. I'm removing this commit before merging.

cargo-dist/src/cli.rs

Gankra · 2023-11-28T19:36:33Z

cargo-dist/src/config.rs

+        if extra_artifacts.is_some() {
+            warn!("package.metadata.dist.extra_artifacts is set, but this is only accepted in workspace.metadata (value is being ignored): {}", package_manifest_path);
+        }


oh interesting, i would have assumed this was package-specific (vaguely recalls some discussion we had about this but forgets the conclusion)

Gankra · 2023-11-28T19:37:36Z

cargo-dist/src/generic_build.rs

+    eprintln!();
+    eprintln!("stdout:");
+    stderr().write_all(&result.stdout).into_diagnostic()?;


this is a weird juxtaposition... but ok i think i see why

This is for consistency with the cargo builds, where we have things which consume the underlying build's stdout. I'd run into cases where allowing the output to go to stdout caused other parts of our build process to fail. (This code is actually from the original generic builds code, just duplicated here.)

Gankra · 2023-11-28T19:40:26Z

cargo-dist/src/tasks.rs

+                        build: extra.build.to_owned(),
+                        artifact: target_path.to_owned(),
+                    }),
+                    checksum: None,


hmm i imagine we'll add a flag for that one day

Yeah, I figured we could talk about what we wanted here and maybe add it later.

cargo-dist/src/tasks.rs

cargo-dist/src/main.rs

cargo-dist/src/tasks.rs

This allows building and packaging arbitrary extra artifacts alongside code tarballs. For example, cargo-dist itself uses it for its own schema metadata JSON.

mistydemeo requested a review from Gankra November 28, 2023 00:09

mistydemeo force-pushed the extra-artifacts branch 7 times, most recently from 79a01bd to dd0890e Compare November 28, 2023 18:37

Gankra approved these changes Nov 28, 2023

View reviewed changes

mistydemeo force-pushed the extra-artifacts branch from dd0890e to a1dd84b Compare November 28, 2023 19:59

Gankra approved these changes Nov 28, 2023

View reviewed changes

cargo-dist/src/main.rs Outdated Show resolved Hide resolved

cargo-dist/src/tasks.rs Outdated Show resolved Hide resolved

mistydemeo force-pushed the extra-artifacts branch 3 times, most recently from cfc695a to a267a0f Compare November 28, 2023 21:55

mistydemeo mentioned this pull request Nov 29, 2023

docs: document extra artifacts #617

Merged

mistydemeo added 3 commits November 29, 2023 09:03

feat: add support for extra artifacts

302f928

This allows building and packaging arbitrary extra artifacts alongside code tarballs. For example, cargo-dist itself uses it for its own schema metadata JSON.

feat(cli): manifest-schema prints to file

e5a578c

feat: use extra artifacts for schema

60f5d13

mistydemeo force-pushed the extra-artifacts branch 3 times, most recently from cf61ea8 to 60f5d13 Compare November 29, 2023 17:30

mistydemeo mentioned this pull request Nov 29, 2023

feat: use extra artifacts instead of append release axodotdev/oranda#667

Merged

mistydemeo merged commit 28c2bd1 into main Nov 29, 2023
14 checks passed

mistydemeo deleted the extra-artifacts branch November 29, 2023 18:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: extra build artifacts #613

feat: extra build artifacts #613

mistydemeo commented Nov 28, 2023 •

edited

Loading

Gankra left a comment

Gankra Nov 28, 2023

mistydemeo Nov 28, 2023

Gankra Nov 28, 2023

Gankra Nov 28, 2023

mistydemeo Nov 28, 2023

Gankra Nov 28, 2023

mistydemeo Nov 28, 2023

feat: extra build artifacts #613

feat: extra build artifacts #613

Conversation

mistydemeo commented Nov 28, 2023 • edited Loading

Gankra left a comment

Choose a reason for hiding this comment

Gankra Nov 28, 2023

Choose a reason for hiding this comment

mistydemeo Nov 28, 2023

Choose a reason for hiding this comment

Gankra Nov 28, 2023

Choose a reason for hiding this comment

Gankra Nov 28, 2023

Choose a reason for hiding this comment

mistydemeo Nov 28, 2023

Choose a reason for hiding this comment

Gankra Nov 28, 2023

Choose a reason for hiding this comment

mistydemeo Nov 28, 2023

Choose a reason for hiding this comment

mistydemeo commented Nov 28, 2023 •

edited

Loading